
METHOD AND SYSTEM FOR STORING CALIBRATION DATA WITHIN 

IMAGE FILES 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[001] This application claims priority from U.S. Provisional Application Ser. No. 
60/270,967 filed 02/22/2001 and entitled METHOD FOR STORING 
CALIBRATION DATA WITHIN IMAGE FILES, incorporated herein by reference. 



[002] This invention relates generally to the field of storing calibration data for a 
probe, and more particularly to a method for storing calibration data within image 
transfer media. 



[003] In certain endoscopes/borescopes, hereinafter referred to as probes, there are 
data associated with the images, such as the calibration parameters for the 
measurement tip and probe that were used to capture the image, along with audio 
comments regarding the captured image, that must be kept with the images. In a 
competitive system, image data, audio data, and calibration data are each stored in 
separate files. This approach allows the audio and/or calibration data easily to 
become separated from the image making features such as off-line measurement and 
audio playback unusable. Embedding the data right in the image solves this problem. 

[004] Graphical overlay data added to images can obscure parts of the image. It is 
generally desirable for this overlay data to be viewable using standard software 
packages, but it is also desirable in some applications to be able to recover the image 
data that has been replaced by the overlay. This invention allows both goals to be 
met. 



[005] Briefly stated, a system and method for storing, within an image transfer 
medium, an image and image-specific data associated with the image includes 
obtaining the image-specific data from a probe such as a borescope or endoscope, 
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obtaining the corresponding image, choosing a specific image transfer medium, 
writing the image to the medium, and writing the image-specific data to a marker in 
the medium. In this manner, storing a combination of image data and one or more of 
system calibration data, overlay replacement data, and audio comment data in a single 
file of either a non-standard file format or a standard file format that does not 
explicitly support the inclusion of these data types is possible. 

[006] According to an embodiment of the invention, a method for storing calibration 
data within image transfer media, includes the step of embedding data specific to a 
measurement system into the image transfer media so that the data is retrievable by a 
custom application directly from the image transfer media, thereby allowing 
re-measurement without using a second transfer media for measurement system 
information. 

[007] According to an embodiment of the invention, a method for storing overlay 
replacement data within image transfer media includes the step of embedding data 
into the image transfer media so that a destructive overlay added to the image is 
visible using a standard image viewer, and image data that was replaced by the 
destructive overlay is reconstituted from the embedded data. 

[008] According to an embodiment of the invention, a method for storing audio data 
along with an image within a standard image transfer media which does not provide 
explicit support for storing audio data includes the step of writing the audio data to a 
marker in the image transfer media such that the image is visible using a standard 
image viewer, while the audio data is retrievable by a custom application. 

[009] According to an embodiment of the invention, a method for storing image data 
and corresponding image-specific data includes the step of storing a combination of 
image data and one or more of system calibration data, overlay replacement data, and 
audio comment data in a single file of either a non-standard file format or a standard 
file format that does not explicitly support the inclusion of these data types. 



[010] According to an embodiment of the invention, a method for storing, within an 
image transfer medium, an image and image-specific data associated with the image 
includes the steps of obtaining the image-specific data; obtaining the image; choosing 
a specific image transfer medium; writing the image to the medium; and writing the 
image-specific data to a marker in the medium. 

[01 1] According to an embodiment of the invention, a system for storing calibration 
data within image transfer media includes means for embedding data specific to a 
measurement system into the image transfer media so that the data is retrievable by a 
custom application directly from the image transfer media, thereby allowing 
re-measurement without using a second transfer media for measurement system 
information. 

[012] According to an embodiment of the invention, a system for storing overlay 
replacement data within image transfer media includes means for embedding data into 
the image transfer media so that a destructive overlay added to the image is visible 
using a standard image viewer, and image data that was replaced by the destructive 
overlay is reconstituted from the embedded data. 

[013] According to an embodiment of the invention, a system for storing audio data 
along with an image within a standard image transfer media which does not provide 
explicit support for storing audio data includes means for writing the audio data to a 
marker in the image transfer media such that the image is visible using a standard 
image viewer, while the audio data is retrievable by a custom application. 

[014] According to an embodiment of the invention, a system for storing image data 
and corresponding image-specific data includes storing a combination of image data 
and one or more of system calibration data, overlay replacement data, and audio 
comment data in a single file of either a non-standard file format or a standard file 
format that does not explicitly support the inclusion of these data types. 



[015] According to an embodiment of the invention, a system for storing, within an 
image transfer medium, an image and image-specific data associated with the image 
includes means for obtaining the image-specific data; means for obtaining the image; 
means for choosing a specific image transfer medium; means for writing the image to 
the medium; and means for writing the image-specific data to a marker in the 
medium. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[016] Fig. 1 shows the encoding process of the present invention; 

Fig. 2 shows the steps to recover data from a JPEG image file according to an 
embodiment of the invention; 

Fig. 3 shows the steps to recover data from a bitmap image file according to an 
embodiment of the invention; and 

Fig. 4 shows the process the system uses to clear an overlay. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[017] The method of the invention could be used in any system where there is a 
graphic overlay added to images that must be removable, or where there are 
non-graphical data related to an image that are required for later use with the image. 
In one application, the method is used to save shadow measurement tip calibration 
data and overlay removal data in bitmap and JPEG images captured using a 
videoprobe remote visual inspection system or an accompanying personal computer 
application. This allows images to have "destructive" overlays that are visible in the 
image using standard image viewing software, but which are removable by a custom 
application to present a clean image to the viewer. Storing tip calibration data in the 
image also allows measurements to be repeated on the image using either the system 
software or a custom PC-based software package. Similarly, audio data, such as 
comments, keywords, or phrases related to the image being viewed, could be included 
in the image file and later recovered. For example, if a person is inspecting the 5 th 
stage of engine type Fl 10 serial number 123456, this information could be stored as 
audio data along with the image being viewed. 



[018] Referring to Fig. 1, the encoding process of the invention is shown using 
calibration data for a borescope or endoscope (hereinafter referred to as a "probe") and 
a JPEG file as an example. In step 10, measurement tip calibration data is read. In 
step 12, the video image from the probe is captured. In step 14, the user identifies the 
specific optical measurement tip being used. The desired measurement, such as, for 
example, measuring the length of a defect observed with the probe, is performed using 
non-destructive overlays in step 16. A replica of the original image data with no 
overlay is made in step 18. Then, in step 20, the overlay is merged destructively into 
the replicated image data. In step 22, a coordinate list of pixel blocks affected by the 
overlay is generated. 

[019] In step 24 the question is asked whether or not JPEG image format is required, 
or whether bitmap format would work. If JPEG format is required, the standard JPEG 
header is written to the file in step 26. The JPEG file format allows for user-defined 
markers to be placed in the file. Each marker can specify up to 64 kilobytes of user 
data to follow. The markers and data are ignored by general image viewers, but can 
be read by application specific viewers. An embodiment of the invention places 
shadow measurement tip calibration parameters in one of these fields, and overlay 
replacement data in two or more others. Specifically, one marker stores a list of the 
coordinates of the 8x8 pixel-blocks in the image that contain overlay data. Another 
marker stores a compressed version of those 8x8 pixel-blocks without the overlay. If 
more than 64 kilobytes are required, additional markers are used. When the image is 
retrieved, these markers and data can be extracted, and the stored 8x8 pixel-blocks can 
be decompressed. They can then replace the corresponding pixel-blocks in the 
decompressed original image, effectively removing the overlay from the image. 
Additional markers could also be used to store audio data. 

[020] The system information is written to the marker in the file in step 28, after 
which the measurement/tip calibration data are written to the marker in the file in step 
30. 8x8 overlay replacement block coordinates are written to the marker in the file in 
step 32. Then an overlay replacement image with all the data values set to zero is 
created in step 34. All 8x8 pixel blocks of original image data affected by the overlay 



are copied into the overlay replacement image in step 36. The overlay image is 
compressed in step 38, and then written to the marker in the JPEG file in step 40. In 
step 42, audio data is optionally written to the marker in the file if present. In step 44, 
the image with the destructive overlay is compressed and written to the JPEG file, 
after which the file is saved in step 60. 

[021] If JPEG format is not required, the standard bitmap header is written to the file 
in step 46. With bitmap images, the shadow measurement tip calibration parameters, 
the 8x8 pixel-block coordinate list, and the non-compressed 8x8 pixel-blocks are 
stored at the end of the file, after the image data. Audio data could also be added to 
the end of the file. General image viewers ignore this additional data, but application 
specific viewers can look for it and extract it. When the image is retrieved, the stored 
8x8 pixel-blocks can replace the corresponding pixel-blocks in the original image, 
effectively removing the overlay from the image. 

[022] In step 48, the image data, including the overlay, is written to the image file. 
The system information is written to the file in step 50. Then, the measurement 
calibration data are written to the file in step 52, after which the overlay replacement 
data coordinates and the data are written to the file in step 54. Audio data is 
optionally written to the file in step 56, after which the file is saved in step 60. 

[023] Referring to Fig. 2, the steps to recover data from a JPEG image file are 
shown. The JPEG file is opened in step 62, after which the main image is 
decompressed in step 64. In step 66, the existence of the system information marker 
is checked. If the marker does not exist, the process ends in step 99. If the marker 
exists, the existence of the calibration data marker is checked in step 68. If the 
calibration data marker exists, the calibration data is read and saved for measurement 
in step 70. The block coordinate list is then read and saved in step 72. In step 74, the 
system checks to see if any blocks are listed, and if not, the process stops in step 99. 
Otherwise, the overlay replacement image is decompressed and saved. 



[024] Referring to Fig. 3, the steps to recover data from a bitmap file are shown. 
The bitmap file is opened in step 82, after which the existence of the system 
information marker is checked in step 84. If the system information marker is not 
present, the process ends at step 99. If the system information marker is present, the 
system looks for the calibration data marker in step 86. If the calibration data marker 
exists, the calibration data is read instep 88 and saved for measurement, then the 
block coordinate list is read and saved in step 90. In step 92, the system checks to see 
if any blocks are listed. If no blocks are listed, the process ends at step 99. 
Otherwise, the block data list is read and saved in step 94. 

[025] Referring to Fig. 4, the process the system uses to clear an overlay is shown. 
In step 95, the system checks to see if a user has issued a "clear overlay" command. If 
so, the system checks in step 96 to see if any blocks are listed for replacement. If not, 
the process ends at step 99. If any blocks are listed for replacement, in step 97 the 
block coordinate list is used to copy 8x8 pixel blocks from the replacement 
data/image into the main image. 

[026] There is a wide variety of image transfer media which can be used for the 
embedded measurement and overlay removal data. For example, the standard image 
transfer media can be digital still images such as JPEG, bitmap, TIFF, PCX etc.; 
digital motion video such as MPEG, AVI, etc.; and analog video using an approach 
similar to closed captioning. With the method of the present invention, the bitmap 
file structure preferably includes: 

(a) Bitmap Header, 

(b) Bitmap image data (with overlay), 

(c) System info section, 

(d) Measurement/tip calibration data section, 

(e) Overlay replacement coordinates/data, and 

(f) Audio comment data section. 

[027] The JPEG file structure preferably includes: 
(a) JPEG Header, 



• # 

(b) System info marker (JFIF Extension), 

(c) Measurement/tip calibration data marker (JFIF Extension), 

(d) Overlay replacement coordinates marker (JFIF Extension), 

(e) Compressed overlay replacement image marker (JFIF Extension), 

(f) Audio comment marker (JFIF Extension), and 

(g) Image data (with overlay). 



[028] The system info section/marker preferably includes: 





(a) Header to identify source and type of data, 




(b) Number of bytes in section, 




(c) Image dimensions, 




(d) Original image source, whether an endoscope system or not, 


w 


(e) System software versions, 




(f) Standard optical distortion (for use in reference-based measurements), 




(g) System serial number, 




(h) Zoom level, 


0 
ru 
ru 


(i) Image horizontally flipped from original or not, 


(j) Video standard of system (NTSC or PAL), and 


0 
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(k) Exposure control mode. 



[029] The measurement/tip calibration data section/marker preferably includes: 

(a) Header to identify source and type of data, 

(b) Number of bytes in section, 

(c) Positions of cursors from measurement screen, 

(d) Type of measurement performed, 

(e) Measurement result, 

(f) Format of tip calibration data, 

(g) Tip type (forward view or side view), 

(h) Tip color code, 

(i) Tip serial number, 

(j) Tip optical distortion, 

(k) Shadow geometry parameters, and 
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(1) Checksum of tip calibration data. 

[030] The JPEG overlay replacement coordinates marker preferably includes: 

(a) Header to identify source and type of data, 

(b) Number of bytes in section, and 

(c) X/Y coordinates of 8x8 pixel blocks affected by overlay. 

[03 1 ] The JPEG overlay replacement data marker preferably includes: 

(a) Header to identify source and type of data, 

(b) Number of bytes in section, and 

(c) Compressed overlay replacement image where all 8x8 pixel blocks affected 
by overlay were filled with the original image data prior to compression. All blocks 
not affected by the overlay are set to values of 0 to allow maximum compression on 
those areas. JPEG compresses images in 8x8 pixel blocks. Information in one block 
does not affect the compression in any other block, so when the two compressed 
images are later uncompressed, the 8x8 blocks from the overlay replacement image 
used to "erase" the overlay are identical to what they would have been in the original 
image had there been no overlay. 

[032] The bitmap overlay data replacement section preferably includes: 

(a) Header to identify source and type of data, 

(b) Number of 8x8 pixel overlay replacement block packets in section, and 

(c) Series of block packets each consisting of horizontal and vertical block 
coordinates followed by 192 bytes of data (8x8 pixels per block, 1 red byte, 1 green 
byte, 1 blue byte per pixel). 

[033] The audio comment data marker/section preferably includes: 

(a) Header to identify source and type of data, 

(b) Number of bytes in section, and 

(c) Audio data. 



• # 

[034] While the present invention has been described with reference to a particular 
preferred embodiment and the accompanying drawings, it will be understood by those 
skilled in the art that the invention is not limited to the preferred embodiment and that 
various modifications and the like could be made thereto without departing from the 
scope of the invention as defined in the following claims. 
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